Encoding/recording device that suspends encoding for video data and sampling for an audio signal in response to a recording pause instruction so as to allow data recorded before and after recording pause to be continuously reproduced

ABSTRACT

An encoding/recording device encodes a video signal according to MPEG, encodes an audio signal with a predetermined method, and multiplexes the encoded video signal and the audio signal to produce a system stream. The encoding/recording device then records the system stream onto a recording medium. On receiving a pause instruction, the encoding/recording device suspends reception of the audio signal and the video signal in synchronization with a timing with which the video signal is received, and thereafter suspends encoding, multiplexing, and recording for the audio signal and the video signal with predetermined timings. On receiving a pause release instruction, the encoding/recording device resumes reception of the audio signal and the video signal in synchronization with a timing with which the video signal is received, and thereafter resumes the encoding, the multiplexing, and the recording with predetermined timings. When suspending the encoding for the video signal, the encoding/recording device keeps information showing how an amount of video data accumulated in a buffer in a virtual decoder changes, and uses the kept information when resuming the encoding.

[0001] This application is based on application No. 11-265575 filed inJapan, the content of which is hereby incorporated by reference.

BACKGROUND OF THE INVENTION

[0002] (1) Field of the Invention

[0003] The present invention relates to an encoding/recording device andan encoding/recording method for compressing and encoding video databased on time correlation properties of the video data, multiplexing thevideo data with audio data, and recording the multiplexed video data andaudio data. More specifically, the present invention relates to atechnology for enhancing a function to suspend recording by theencoding/recording device.

[0004] (2) Description of the Prior Art

[0005] In recent years, increasing amount of information is digitized.Especially, more and more sound and images are digitized since digitalinformation suffers no degradation with the passage of time and isrelatively easy to be processed. Hereafter, sound and images in digitalformat is collectively called “AV (audio-video) data”.

[0006] MPEG (Moving Picture Expert Group, including MPEG-2 in thisspecification) is an international standard used to compress AV data torecord it effectively.

[0007] MPEG for video data uses a compression method based on timecorrelation properties between different pictures in addition to aconventionally used compression method based on discrete cosinetransform (DCT). The compression method based on the time correlationproperties achieves a high compression rate by representing one pictureas differential data between the picture and other similar pictures tothis picture which are reproduced before and after this picture.However, with this MPEG based on the time correlation properties,presenting order and decoding order for pictures are different, andtherefore it is necessary to record, or decode and store a picture thatis referred to for encoding another picture, prior to the other picturethat refers to this picture. For MPEG, a picture that is referred to forencoding of another picture is called an I picture (intraframepredictively encoded picture) or a P picture (interframe predictivelyencoded picture). A picture that is encoded referring to another picture(I or P picture) is called a B picture (bi-directionally predictivelyencoded picture).

[0008] Video data is a plurality of sets of still image data per unittime (each set of still image data is hereafter called a video frame),and therefore video data usually contains similar images. As MPEG canprovide a higher compression rate for video data containing more imagessimilar to one another, MPEG is suitable for compression of video data.

[0009] MPEG can effectively compress data by providing a differentcompression rate for each image and dynamically assigning encoding bitsto the image in accordance with its complexity.

[0010] Audio data, on the other hand, has a smaller size than videodata, and therefore a different compression method than used for thevideo data is usually used.

[0011] For instance, a DVD recorder that records AV data onto a DVD-RAMaccording to MPEG allows a user to select whether to compress audiodata. When selecting that the AV data should be compressed, the user canfurther select whether MPEG Audio or Dolby AC3 should be used as acompression method. The DVD recorder then encodes the audio data usingthe selected compression method. When the user selects that nocompression is performed for the audio data, LPCM (linear pulse codemodulation) is performed for the audio data. The DVD recorder thenencodes and compresses video data according to MPEG, multiplexes theencoded video data and audio data into a piece of MPEG System streamaccording to MPEG System, and records the piece of MPEG System stream.

[0012] With MPEG System, audio data and video data, which have beenencoded and compressed, are divided into audio packets and video packetsthat have predetermined sizes, and time-division multiplexed into MPEGSystem stream. Hereafter, the terms “audio data” and “video data” areused to represent audio data and video data that have been encoded andcompressed. MPEG System stream has a hierarchical structure composed ofa pack and a packet, with one pack being composed of one or morepackets. For instance, a pack recorded on a DVD-RAM is composed of onepacket. For ease of explanation, one pack is assumed to be composed ofone packet in this specification, as is the case with a pack recorded ona DVD-RAM.

[0013]FIG. 1 shows a construction of a pack and a packet generatedaccording to MPEG System.

[0014] Each packet is 2 KB, and contains a pack header 11, a packetheader 12, and a payload 13.

[0015] The pack header 11 contains an SCR (system clock reference) thatshows a time at which the pack should be. inputted to a video buffer oran audio buffer in an MPEG decoder.

[0016] The packet header 12 contains the following information: a streamID that identifies the content of the payload 13; a DTS (Decoding TimeStamp) showing a decoding start time; and a PTS (Presentation TimeStamp) showing a presentation time. Note that an audio pack does notcontain a DTS since audio data is decoded and presented almostsimultaneously.

[0017] The payload 13 is audio data or video data.

[0018] Audio data is usually divided into audio packets that eachcontain audio data corresponding to one audio frame, and therefore alarge-capacity audio buffer is not required by the MPEG decoder. As withvideo data, however, video frames have different sizes, and thedifferences in the size between different video frames are very large.For instance, video data corresponding to one video frame may be dividedinto a plurality of video packets. Accordingly, an MPEG decoder isrequired to have a video buffer that has at least the same size as avideo frame of the largest size. Packs are positioned in MPEG Systemstream in order of an SCR, the earliest SCR first.

[0019]FIG. 2 is a diagram showing a standard decoder for MPEG Systemstream.

[0020] This MPEG decoder comprises the following elements: an STC(system time clock) 21 that generates a system time based on which theMPEG decoder operates, a demultiplexer 22 that separates system streaminto audio packets and video packets based on a stream ID of eachpacket; a video buffer 23 that temporarily buffers video data; a videodecoder 24 that decodes video data; a reordering buffer 25 thattemporarily stores video data to be referred to by other video data; aswitch 26 that is used to adjust output order of video data; an audiobuffer 27 that temporarily buffers audio data; and an audio decoder 28that decodes audio data.

[0021] The following describes decoding operations by this MPEG decoder.

[0022] A pack is extracted to be inputted to the demultiplexer 22 when asystem time generated by the STC 21 agrees with an SCR written in thepack. The demultiplexer 22 then refers to a stream ID of the inputtedpack and sends a packet in the pack to either the video buffer 22 or theaudio buffer 27 accordingly. The video buffer 23 accumulates payloads ofpackets sent by the demultiplexer 22 and manages a DTS and a PTS of eachpacket. The video decoder 24 reads video data that has a DTS equal to acurrent system time from the video buffer 23. This read video datacorresponds to one video frame. The video decoder 24 then decodes theread video data. Following this, video data (i.e., an I picture or a Ppicture) which is referred to for encoding of other pictures istemporality buffered by the reordering buffer 25, and selectivelyoutputted in accordance with a PTS via the switch 26. The video decoder24 decodes video data (i.e., B picture) that is encoded referring toother pictures, and outputs it immediately. On receiving audio data thatis a payload of each audio packet from the demultiplexer 22, the audiobuffer 27 buffers it and manages a PTS in the audio packet. The audiodecoder 28 reads audio data that has a PTS equal to a current systemtime from the audio buffer 37. This read audio data corresponds to oneaudio frame. The audio decoder 28 then decodes the read audio data.

[0023] In order to present images without delays, MPEG defines that anMPEG decoder starts decoding video data only after the video buffer 23has become full. This generates a time lag between a start ofaccumulation of packets in the video buffer and a start of decoding forvideo data. This time lag is called “vbv_delay” in MPEG. MPEG alsodefines a capacity of a video buffer as 224KB, and data of a sizeexceeding this capacity is not allowed to be buffered. MPEG furtherdefines that the video buffer cannot buffer the same data for one secondor longer.

[0024] To control a video buffer in accordance with MPEG in this way, anMPEG encoder assigns an SCR and a DTS to each pack appropriately whensuch packs are recorded.

[0025] In this way, video data has to be inputted to a video buffer acertain time before the video data is presented, while audio data has tobe inputted to an audio buffer only shortly before its presentation.Accordingly, when video data and audio data should be presentedsimultaneously, the video data is multiplexed in prior to the audiodata.

[0026] When a sequence of images is recorded after one recording foranother sequence has been completed, the aforementioned time lag“vbv_delay” is generated between the two sequences, which prevents thetwo sequences from being continuously reproduced. This can happen, forinstance, when a program and commercials are broadcasted and only theprogram is recorded without the commercials being recorded, or whendifferent sequences, which are not consecutive, of images are taken by adigital video camera.

[0027] When packets corresponding to different sequences, which are notconsecutive, are joined together to prevent the vbv_delay from beinggenerated, however, a video buffer in the MPEG decoder can no longer beused in accordance with MPEG standard, and therefore may suffer abreakdown.

[0028]FIG. 3A shows transition of a size of video data buffered in avideo buffer in an MPEG decoder used when video data for a sequence ofconsecutive images is reproduced. FIG. 3B shows transition of a size ofvideo data buffered in the video buffer when video packets, which arenot consecutive and have been joined together, are reproduced.

[0029] In FIG. 3B, the video buffer overflows at time t3 as a result ofnon-consecutive packets composed of packets corresponding to a periodbefore time t1 in FIG. 3A and packets corresponding to a period aftertime t2 being joined together by discarding packets corresponding to aperiod from t1 to t2.

[0030] Joining non-consecutive packets without special considerationsbeing given also causes the following problems.

[0031] To simultaneously present sound and images, an audio pack ismultiplexed into MPEG System stream after a period equal to “vbv_delay”has passed since a video pack was multiplexed into the MPEG Systemstream. As a result, if video packets and audio packets corresponding toa certain period in MPEG System stream are discarded, and packets beforeand after the certain period are joined together, sound corresponding toimages immediately before the joined part of the stream are lost whilesounds corresponding to the discarded images remain.

[0032] Secondly, since an audio frame and a video frame have differentframe generation frequencies, deleting certain audio frames and videoframes in units of respective frames results in generating a time lag,the so-called “lip sync (synchronization) lag”, between sound and imagesfor frames that follow the deleted frames. For instance, Dolby DigitalAC-3 compresses audio data for a DVD-RAM as audio frames whose framegeneration frequency is 32 msec although a video frame has a framegeneration frequency of 33.3667 msec. Accordingly, the lip sync lag willalmost certainly occur if certain audio frames and video frames aredeleted in units of respective frames.

[0033] Lastly, when two non-consecutive audio frames are joined togetherafter certain audio frames are deleted, the two non-consecutive audioframes often do not have similarities. As a result, noise is generatedwhen these audio frames are inputted to an audio buffer to bereproduced.

[0034] Accordingly, it is not appropriate to delete certain packets orframes from MPEG System stream and to join remaining packets or framestogether without special considerations being given.

SUMMARY OF THE INVENTION

[0035] The present invention is made in view of the above problems, andaims to provide an encoding/recording device and an encoding/recordingmethod for consecutively recording different pieces of AV stream, whichare not consecutive in terms of time, without causing malfunctions tooccur in the above video buffer in the MPEG decoder, and to provide arecording medium storing a program to have a computer perform theencoding/recording method.

[0036] The present invention also aims to provide an encoding/recordingdevice, an encoding/recording method, and a recording medium storing aprogram to have a computer perform the encoding/recording method forrecording non-consecutive AV data in a manner that allows the recordednon-consecutive AV data to be reproduced without troubles relating tosound, such as unnecessary noise and a time lag between images and soundlike “lip sync” lag, being involved.

[0037] In order to achieve the above objects, the present inventionrelates to an encoding/recording device that receives and encodes avideo signal and an audio signal, multiplexes the encoded video signaland audio signal to produce a system stream, and records the systemstream on a recording medium. The encoding/recording device includes: avideo data generating unit operable to (a) estimate an amount of data,which occupies a buffer in a decoder when the decoder decodes theencoded video signal (b) store how the amount of data changes over timeas buffer information, (c) receive a video signal, (d) encode thereceived video signal to generate video data in a manner that preventsthe decoder from overflowing and underflowing, and (e) update the bufferinformation whenever video data is generated, an audio data generatingunit operable to receive an audio signal, and encode each part of thereceived audio signal to generate audio data, each part having apredetermined size; a multiplexing/recording unit operable to multiplexthe generated video data and audio data, and record the multiplexedvideo data and audio data onto the recording medium; a pause controllingunit operable to specify a pause timing on receiving a pause instructionthat suspends recording by the encoding/recording device from a userduring the recording, and have the video data generating unit, the audiodata generating unit, and the multiplexing/recording unit suspendoperations; and a pause release controlling unit operable to specify apause release timing on receiving a pause release instruction from theuser, and have the video data generating unit, the audio data generatingunit, and the multiplexing/recording unit resume the operations. Undercontrol of the pause controlling unit, the video data generating unitsuspends reception of the video signal with the specified pause timingwithout resetting the stored buffer information. Under control of thepause release controlling unit, the video data generating unit resumesthe reception the video signal with the specified pause release timingand resumes encoding the received video signal based on the storedbuffer information. Under control of the pause controlling unit, theaudio data generating unit suspends reception of the audio signal withthe specified pause timing. Under control of the pause releasecontrolling unit, the audio data generating unit resumes receiving theaudio signal with the specified pause release timing, and resumesencoding the received audio signal. The multiplexing/recording unitsuspends multiplexing and recording with a predetermined timing that isequal to or later than the specified pause timing under control of thepause controlling unit, and resumes the multiplexing and the recordingwith a predetermined timing that is equal to or later than the specifiedpause release timing under control of the pause release controllingunit.

[0038] When the user records non-consecutive pieces of AV stream, usingthe above suspension function, the recorded pieces of AV stream can beconsecutively and “seamlessly” reproduced with no intervals beinggenerated between the pieces of AV stream. In addition, the video bufferin the MPEG decoder will not be broken even when AV data, which has beenrecorded after the recording pause, is reproduced.

[0039] Here, the video data generating unit may encode the receivedvideo signal according to MPEG (Moving Picture Expert Group).

[0040] For this construction, the video data generating unit canestimate an amount of data that occupies a buffer in an MPEG decoder andencode the received video signal according to MPEG in a manner thatprevents the buffer from underflowing and overflowing.

[0041] Here, the encoding/recording device may further include a timerthat generates a reference time at least when the encoding/recordingdevice operates and suspends recording. Every first predetermine period,the video data generating unit receives a video signal that correspondsto one video frame, each first predetermined period being based on areference time generated by the timer. The pause timing specified by thepause controlling unit is synchronous to a boundary of two firstpredetermined periods, and the pause release timing specified by thepause release controlling unit is synchronous to a boundary of two firstpredetermined periods.

[0042] For this construction, input of the video signal and the audiosignal is suspended in synchronization with a timing with which aboundary of two video frames appears. As a result, recording can besuspended for each video frame, and no time lags are generated betweenreproduced images and sound which were recorded after the suspension wasreleased.

[0043] Here, under control of the pause controlling unit, themultiplexing/recording unit may suspend the multiplexing and therecording after a second predetermined period has passed since thespecified pause timing, and may hold video data and audio data whichhave not been multiplexed and recorded. Under control of the pauserelease controlling unit, the multiplexing/recording unit may resume themultiplexing and the recording by multiplexing and recording the heldvideo data and audio data after the second predetermined period haspassed since the specified pause release timing.

[0044] For this construction, the multiplexing/recording unit suspendsthe multiplexing and the recording after the second predetermined periodhas passed since the specified pause timing, and holds video data andaudio data that have not been multiplexed and recorded at this point. Asa result, the present encoding/recording device can correctly record AVdata even after the suspension is released.

[0045] Here, after a third predetermined period has passed since thespecified pause timing, the video data generating unit may completeencoding for the video signal which has been received before thespecified pause timing. The second predetermined period for themultiplexing/recording unit may be a sum of the first predeterminedperiod and the third predetermined period.

[0046] For this construction, the multiplexing/recording unit suspendsthe recording and the multiplexing after a. time, which has the sameduration as a frame generation cycle for video data, has passed sincegeneration of video data was completed, and resumes the recording andthe multiplexing after a time having the same duration as the framegeneration cycle has passed since when generation of video data wasresumed. As a result, it becomes possible to make a period for which themultiplexing/recording unit suspends the multiplexing and the recordingapproximately equal to a period for which reception of the audio signaland the video signal is suspended.

[0047] Here, the encoding/recording device may further include a firststop controlling unit operable to specify a stop timing on receiving astop instruction from the user while the encoding/recording deviceoperates, and have the video data generating unit, the audio datagenerating unit, and the multiplexing/recording unit stop operations.Under control of the first stop controlling unit, the video datagenerating unit may stop receiving the video signal with the specifiedstop timing, and encode the video signal received before the stop timingto generate video data. Under control of the first stop controllingunit, the audio data generating unit may stop receiving the audio signalwith the specified stop timing, encode each part of the predeterminedsize contained in the received audio signal to generate audio data, andabandon, if the received audio signal contains a subpart that is smallerthan the predetermined size, the subpart. Under control of the firststop controlling unit, the multiplexing/recording unit may multiplex andrecord all the audio data and video data that have been generated.

[0048] This construction allows the present encoding/recording device tocompletely stop its operations.

[0049] Here, the encoding/recording device may further include: a secondstop controlling unit operable to specify a stop timing on receiving astop instruction from the user while the encoding/recording devicesuspends recording in response to a recording pause instruction, and tohave the video data generating unit, the audio data generating unit, andthe multiplexing/recording unit stop operations. Under control of thesecond stop controlling unit, the video data generating unit may encodethe video signal received before the specified pause timing to generatevideo data. Under control of the-second stop controlling unit, the audiodata generating unit may encode each part of the audio signal receivedbefore the specified pause timing to generate audio data, and abandons,if the received audio signal contains a subpart, the subpart. Undercontrol of the second stop controlling unit, the multiplexing/recordingunit may multiplex and record the held video data and audio data, andthe generated audio data and video data.

[0050] This construction allows the present encoding/recording device toproperly stop after the suspension.

[0051] Here, the audio data generating unit may contain: an audio signalsampling unit operable to receive the audio. signal, sample the receivedaudio signal to generate sets of sampled data that each correspond toone audio sampling cycle, each set of sampled data being generated everyfourth predetermined period that is different from the firstpredetermined period, and suspend reception of the audio signal with thespecified pause timing; an encoding unit operable to receive a set ofsampled data from the audio signal sampling unit every fourthpredetermined period, and encode the received set of sampled data togenerate audio data corresponding to one audio sampling cycle; afractional data holding unit operable to hold a set of sampled datawhich corresponds to a,time shorter than the fourth predetermined periodwhen the audio signal sampling unit suspends the reception of the audiosignal. The audio signal sampling unit may resume receiving the audiosignal with the specified pause release timing, sample the receivedaudio signal to generate a set of sampled data, join the generated setof sampled data and the held set of sampled data together to produce aset of sampled data corresponding to one audio sampling cycle, outputthe produced set of sampled data, and thereafter output a set of sampleddata corresponding to one audio sampling cycle every fourthpredetermined period.

[0052] For this construction, audio data smaller than a predeterminedsize can be held, so that no time lags occur between images and soundrecorded after the suspension is released.

[0053] Here, under control of the pause controlling unit, themultiplexing/recording unit may suspend the multiplexing and therecording after having multiplexed and recorded all the video datagenerated by the video data generating unit.

[0054] With this construction, the present encoding/recording devicesuspends recording only after having recorded all the generated videodata. Accordingly, video data can be correctly multiplexed and recordedafter the suspension is released.

[0055] Here, when suspending the multiplexing and the recording, themultiplexing/recording unit may hold audio data that has not beenmultiplexed and recorded. Under control of the pause release controllingunit, the multiplexing/recording unit may resume the multiplexing andthe recording by starting multiplexing and recording (a) the held audiodata and (b) video data generated after the specified pause releasetiming.

[0056] For this construction, audio data, to which a later multiplexingorder is applied, is held when the recording and the multiplexing aresuspended. When the suspension is released, the held audio data aremultiplexed after video data, which is generated after the suspension isreleased, are multiplexed. As a result, when images and sound recordedafter the suspension are reproduced, a time lag will not be generatedbetween the images and sound.

[0057] Here, the encoding/recording device may further include a firststop controlling unit operable to specify a stop timing on receiving astop instruction from the user while the encoding/recording deviceoperates, and to have the video data generating unit, the audio datagenerating unit, and the multiplexing/recording unit stop operations.Under control of the first stop controlling unit, the video datagenerating unit may stop receiving the video signal with the specifiedstop timing, and encode the video signal received before the stop timingto generate video data. Under control of the first stop controllingunit, the audio data generating unit may stop receiving the audio signalwith the specified stop timing, encode each part of the predeterminedsize contained in the received audio signal to generate audio data, andabandon, if the received audio signal contains a subpart that is smallerthan the predetermined size, the subpart. Under control of the firststop controlling unit, the multiplexing/recording unit may multiplex andrecord all the audio data and video data that have been generated.

[0058] For this construction, the encoding/recording device can stopafter a recording operation.

[0059] Here, the multiplexing/recording unit may further contain asecond stop controlling unit operable to specify a stop timing onreceiving a stop instruction from the user while the encoding/recordingdevice suspends recording, and have the video data generating unit, theaudio data generating unit, and the multiplexing/recording unit stopoperations. The audio data generating unit may abandon a subpart if thereceived audio signal contains the subpart. Under control of the secondstop controlling unit, the multiplexing/recording unit may multiplex andrecord all the held audio data.

[0060] For this construction, the encoding/recording device cancorrectly stop after the recording pause.

[0061] Here, the audio data generating unit may contain: an audio signalsampling unit operable to receive the audio signal, sample the receivedaudio signal to generate sets of sampled data that each correspond toone audio sampling cycle, each set of sampled data being generated everyfourth predetermined period that is different from the firstpredetermined period, and suspend reception of the audio signal with thespecified pause timing; an encoding unit operable to receive a set ofsampled data from the audio signal sampling unit every fourthpredetermined period, and encode the received set of sampled data togenerate audio data corresponding to one audio sampling cycle; afractional data holding unit operable to hold a set of sampled datawhich corresponds to a time shorter than the fourth predetermined periodwhen the audio signal sampling unit suspends the reception of the audiosignal. The audio signal sampling unit may resume receiving the audiosignal with the specified pause release timing, sample the receivedaudio signal to generate a set of sampled data, join the generated setof sampled data and the held set of sampled data together to produce aset of sampled data corresponding to one audio sampling cycle, outputthe produced set of sampled data, and thereafter output a set of sampleddata corresponding to one audio sampling cycle every fourthpredetermined period.

[0062] For this construction, audio data smaller than a predeterminedsize is held during the recording pause, so that no time lag occursbetween images and sound recorded after the suspension is released.

[0063] Here, the audio data generating unit may contain a muting unitoperable to lower a sound intensity of the audio signal before receptionof the audio signal is suspended, and to restore the sound intensitybefore the reception of the audio signal is resumed.

[0064] With this construction, unnecessary noise is not generated when arecorded part corresponding to the recording pause is reproduced.

[0065] Here, the video signal received by the video data generating unitmay correspond to a plurality of video frames. The video data generatingunit may encode the received video signal using time correlationproperties between some of the plurality of video frames, and generate aplurality of GOPs (groups of pictures). Each GOP may be video datacorresponding to a plurality of video frames that are encoded withreference only to video frames in the same GOP. When suspending thereception of the video signal, the video data generating unit maycomplete generation of a GOP.

[0066] For this construction, generation of a GOP is completedimmediately before the recording is suspended. Accordingly, a randomaccess can be performed to the recorded MPEG System stream, such as whena reproduction in fast-forward mode or in fast-rewind mode, or areproduction from a midpoint of the MPEG System stream is performed.

[0067] Here, the multiplexing/recording unit may generate system streamby multiplexing audio data and video data, and the system stream may becomposed of a plurality of video object units (VOBUs). Each VOBU may becomposed of at least one GOP and audio data related to the at least oneGOP, and have a representation time shorter than a predetermined time.When suspending the recording, the multiplexing/recording unit maycomplete generation of a VOBU using video data, which has been generatedby the video data generating unit using the video signal received beforethe specified pause timing, and audio data corresponding to a decodingorder that is earlier than the decoding order corresponding to the usedvideo data. The decoding order may be an order in which the decoderdecodes the video data and the audio data. When resuming the recording,the multiplexing/recording unit may make video data, which has beengenerated by the video data generating unit immediately after thespecified pause release timing, video data placed first in a VOBU thatfollows the VOBU generated when the recording is suspended.

[0068] For this construction, generation of a VOBU (video object unit)can be completed immediately before the recording is suspended, whichenhances ease of operations of the present encoding/recording devicewhen AV data is reproduced.

[0069] Here, the multiplexing/recording unit may generate a video object(VOB) composed of a plurality of VOBUs, and the VOB may contain arecording region into which a recording start time for the VOB should bewritten. When suspending the recording, the multiplexing/recording unitmay complete generation of a VOB. When resuming the recording, themultiplexing/recording unit may newly generate a VOBU and a VOB, placethe generated VOBU at a start of the generated VOB, and write a time atwhich the recording is resumed as a recording start time into arecording region in the generated VOB.

[0070] With this construction, generation of a VOB (video object) can becompleted immediately before the recording is suspended, so that asuitable recording time can be written into a VOB recorded after therecording pause is released.

[0071] Here, when suspending the reception of the video signal, thevideo data generating unit may attach a sequence end code to an end ofvideo data which has been generated from the received video signal. Thesequence end code may indicate an end of video stream that is video datacorresponding to a plurality of video frames.

[0072] For this construction, the video data generating unit inserts asequence end code into an end of video data that is generatedimmediately before the recording is suspended. As a result, AV data canbe recorded in conformity with the DVD-RAM standard.

[0073] Here, when suspending the recording, the multiplexing/recordingunit may attach a sequence end code to an end of video data, which hasbeen generated by the video data generating unit using the video signalreceived before the specified pause timing. The sequence end code mayindicate an end of video stream that is video data corresponding to aplurality of video frames.

[0074] With this construction, the multiplexing/recording unit inserts asequence end code into an end of video data that is recorded immediatelybefore the recording is suspended. This allows AV data to be recorded inconformity with the DVD-RAM standard.

BRIEF DESCRIPTION OF THE DRAWINGS

[0075] These and the other objects, advantages and features of theinvention will become apparent from the following description thereoftaken in conjunction with the accompanying drawings which illustrate aspecific embodiment of the invention.

[0076] In the drawings:

[0077]FIG. 1 shows a construction of a pack and a packet generatedaccording to MPEG System.

[0078]FIG. 2 is a diagram showing a standard decoder for MPEG Systemstream.

[0079]FIG. 3A shows transition of a size of video data buffered in avideo buffer in an MPEG decoder used when video data for a sequence ofconsecutive images is reproduced.

[0080]FIG. 3B shows transition of a size of video data buffered in thevideo buffer when video packets, which are not consecutive and have beenjoined together, are reproduced.

[0081]FIG. 4 shows a construction of a DVD recorder of the presentembodiment according to the present invention.

[0082]FIG. 5 shows a detailed construction of an encoding unit 104 inthe DVD decoder.

[0083]FIG. 6 shows the processing by the encoding unit 104 when it hasreceived a recording pause instruction during recording, when therecording pause is released, and when the encoding unit 104 has receiveda recoding stop instruction.

[0084]FIG. 7 is a timing chart that shows input and output timing ofvideo data and audio data when the recording is suspended and then therecoding suspension is cleared.

[0085]FIG. 8 shows, as one example, how video data and audio data areused when a recording pause instruction is issued.

[0086]FIG. 9 shows how video data and audio data are used when arecording pause instruction is issued, as another example.

[0087]FIG. 10 shows how video data and audio data are used when arecording pause instruction is issued, as another example.

[0088]FIG. 11 shows, as another example, how video data and audio dataare used before and after a recording pause.

[0089]FIG. 12 shows how video data and audio data are used when arecording stop instruction is issued during recording.

[0090]FIG. 13 shows how video data and audio data are used when arecording stop instruction is issued while recording is suspended asshown in FIG. 8.

[0091]FIG. 14 shows how video data and audio data are used when arecording stop instruction is issued while recording is suspended asshown in FIG. 10.

[0092]FIG. 15 shows a sequence end code inserted into the end of a GOPgenerated immediately before the suspension.

[0093]FIG. 16 shows a state in which a system encoder 208 inserts asequence end code into an end of a GOP generated immediately before arecording pause.

DESCRIPTION OF THE PREFERRED EMBODIMENT

[0094] 1. Overview

[0095] The present embodiment relates to an encoding/recording device.When the user presses a pause key or the like on the presentencoding/recording device, a recording pause instruction is issued. Theencoding/recording device then suspends compression and encoding forvideo data simultaneously with sampling for an audio signal with atiming conforming to a video frame frequency. AV data recorded by thepresent encoding/recording device in this way can be continuouslypresented by an MPEG recorder without a time lag between images andsound being generated and without causing a video buffer in the MPEGrecorder to malfunction.

[0096] 2. Overall Construction

[0097]FIG. 4 shows a construction of the encoding/recording device(hereafter called a DVD recorder) of the present embodiment.

[0098] The DVD recorder comprises a user interface (U/I) unit 101, asystem controlling unit 102, an inputting unit 103, an encoding unit104, an outputting unit 105, a decoding unit 106, and a track buffer107, and a drive unit 108.

[0099] The U/I unit 101 is achieved by switches provided on the DVDrecorder and a remote controller, for instance. The U/I unit 101presents information to the user, receives an instruction from the user,and sends it to the system controlling unit 102.

[0100] The system controlling unit 102 is achieved by a microcomputerand a recording medium storing a control software program, for instance.The system controlling unit 102 receives the user instruction from theU/I unit 101, interprets the user instruction, and controls other units.

[0101] The inputting unit 103 is achieved by a digital video camera witha microphone, or a TV tuner, for example, and outputs a video frame andan audio signal to the encoding unit 104.

[0102] The encoding unit 104 includes the following elements: a videoencoder that encodes an inputted video frame into video data; an audioencoder that performs sampling for the inputted audio signal to generatean audio frame, and encodes the audio frame to generate audio data; anda system encoder that multiplexes the generated video data and audiodata into MPEG System stream.

[0103] The outputting unit 106 is achieved by a digital TV set, forinstance, and outputs images and sound based on a video frame and anaudio frame that has been generated.

[0104] The decoding unit 106 includes the following elements: a systemdecoder that specifies a type of each packet in MPEG System stream tooutput the packet accordingly; an audio decoder that receives audio data(a packet) from the system decoder and decodes the audio data togenerate an audio signal; and a video decoder that receives video data(a packet) from the system decoder and decodes the video data togenerate a video signal.

[0105] The track buffer 107 buffers MPEG System stream to absorb a timelag between the output/input by the encoding unit 104 and theoutput/input by the drive unit 108, and a time lag between theoutput/input by the decoding unit 106 and the output/input by the driveunit 108.

[0106] The drive unit 108 is a disk drive that controls a write and aread of MPEG System stream onto and from a DVD-RAM disc.

[0107] The following describes recording operations by the DVD recorderin FIG. 4.

[0108] The U/I unit 102 receives a recording start instruction from theuser, and sends it to the system controlling unit 102.

[0109] The system controlling unit 102 then interprets the recordingstart instruction, and instructs the encoding unit 104 to startencoding.

[0110] On receiving the recording start instruction, the encoding unit104 starts receiving a video frame and an audio signal from theinputting unit 103, generates MPEG System stream, and sends it to thetrack buffer 107.

[0111] When the track buffer 107 has accumulated MPEG System stream of apredetermined size, the system controlling unit 102 has the drive unit108 to write the accumulated MPEG System stream onto the DVD-RAM disc.

[0112] If the U/I unit 101 receives a recording end instruction from theuser at this point, the U/I unit 101 sends the instruction to the systemcontrolling unit 102.

[0113] The system controlling unit 102 then interprets the recording endinstruction, and instructs the encoding unit 104 to end the recording.

[0114] On receiving this recording end instruction, the encoding unit104 stops receiving a video frame and an audio signal in conformity witha frame generation cycle for video frames, and generates MPEG Systemstream which includes an audio frame produced lastly using the receivedaudio signal. The encoding unit 104 then sends the generated MPEG Systemstream to the track buffer 107, and notifies the system controlling unit102 of the recording end.

[0115] On receiving the recording end notification, the systemcontrolling unit 102 has the drive unit 108 record the MPEG Systemstream from the track buffer 107 to the DVD-RAM disc. The systemcontrolling unit 102 also generates management information such as VideoObject Information (VOBI) and Cell Information, and has the drive unit108 record the generated management information onto the DVD-RAM disc.

[0116] If the user instructs, while the recording is performed, tosuspend recording by pressing a pause key or the like on the DVDrecorder, this recording pause instruction is sent to the U/I interfaceunit 101. The U/I interface unit 101 then sends the recording pauseinstruction to the system controlling unit 102.

[0117] The system controlling unit 102 then interprets the recordingpause instruction,.and instructs the encoding unit 104 to suspendencoding.

[0118] When receiving the encoding suspension instruction, the encodingunit 104 suspends reception of video frames, as a video framecorresponding to a video signal for which sampling was being performedwhen the encoding unit 104 received the encoding suspension instructionas the last video frame. As soon as the sampling for the last videoframe is completed, the encoding unit 104 also suspends sampling for anaudio signal, and generates MPEG System stream while completinggeneration of one Group of Pictures (GOP). Following this, the encodingunit 102 notifies the system controlling unit 102 of the encoding end.

[0119] On receiving the encoding end notification, the systemcontrolling unit.102 has the drive unit 108 suspend a write of MPEGSystem stream from the track buffer 107 onto the DVD-RAM disc.

[0120] The above “GOP” is a group of pictures encoded with reference toone another, and is defined for MPEG video stream as a unit to which arandom access can be performed such as when a reproduction infast-forward mode or in fast-rewind mode, or a reproduction from amidpoint of MPEG System stream is performed. Such a random access withineach GOP is possible because each picture within the GOP only refers topictures of the same GOP.

[0121] Following this, when receiving a user instruction that releasesthe recording pause, the U/I unit 101 sends a pause release instructionto the system controlling unit 102.

[0122] The system controlling unit 101 then interprets the pause releaseinstruction, and instructs the encoding unit 104 to release therecording pause.

[0123] On receiving the pause release instruction, the encoding unit 104resumes reception of a video frame. As soon as the sampling for thisvideo frame starts, the encoding unit 104 starts sampling for an audiosignal, and notifies the system controlling unit 102 of the resumptionof these operations.

[0124] On receiving this notification, the system controlling unit 102has the drive unit 108 record MPEG System stream buffered in the trackbuffer 107 onto the DVD-RAM disc.

[0125] The reproduction processing by the present DVD recorder will notbe described as it is not the essence of the present invention.

[0126]FIG. 5 shows a detailed construction of the encoding unit 104.

[0127] The encoding unit 104 includes a control I/F unit 201, a videoencoder 202, a video buffer 203, an audio muting unit 204, a samplingunit 205, an audio encoder 206, an audio buffer 207, a system encoder208, an end code requesting unit 209, a video encoder pause unit 210, asampling pause unit 211, an audio encoder pause unit 212, and a systemencoder pause unit 213.

[0128] The control I/F unit 201 controls other elements of the encodingunit 104, and information transfer between the system controlling unit102 and the other elements.

[0129] The video encoder 202 calculates an assumed amount of data to beaccumulated in a virtual buffer of a virtual decoder (i.e., VideoBuffering Verifier (VBV)) defined in MPEG, and stores calculated amountsas VBV buffer transition information. The video encoder 202 receives avideo frame, encodes the received video frame into video data, andupdates the VBV buffer transition information.

[0130] The video buffer 203 buffers video data generated by the videoencoder 202.

[0131] In response to a recording pause instruction, the audio mutingunit 204 starts muting an audio signal to be inputted to the samplingunit 205 by decreasing a sound intensity to around zero before thesampling unit 205 suspends the sampling for the audio signal. When therecording pause is released, the audio muting unit 205 restores theoriginal sound intensity after the sampling unit 205 has resumed thesampling for the audio signal.

[0132] The sampling unit 205 receives an audio signal, and performssampling for the audio signal to generate and output one audio framecorresponding to one sampling cycle in each audio frame generationcycle.

[0133] The audio encoder 206 receives an audio frame from the samplingunit 205, and encodes it into audio data.

[0134] The audio buffer 207 buffers audio data generated by the audioencoder 206.

[0135] Based on information regarding a size of each video frame thathas been encoded by the video encoder 202 as a picture, the systemencoder 208 obtains, from the video buffer 203, video data having a sizeequal to that of one video pack, and generates a video pack. Thisinformation is sent by the video encoder 202. The system encoder 208also obtains, from the audio buffer 207, an audio frame of a size thatis suitable to have one audio pack generated, based on informationregarding a playback time of the audio frame. Based on the VBV buffertransition information, the system encoder 208 specifies, for each videopack, a DTS value and an SCR value, which is information regarding atime at which the video pack is multiplexed so as to allow a videobuffer of an MPEG decoder to operate in conformity with MPEG standard.Following this, the system encoder 208 multiplexes the generated audiopacks and video packs to generate MPEG System stream in which thesepacks are positioned in appropriate order, while writing an SCR into apack header of each pack.

[0136] The end code requesting unit 209 requests the video encoder 202to generate a sequence end code and insert it into the end of a GOP.

[0137] The video encoder pause unit 210 has the video encoder 202suspend generation of video data when a recording pause instruction isissued. The video encoder pause unit 210 also instructs the videoencoder 202 to complete generation of one GOP, with a video frame, whichcorresponds to a video signal for which sampling was being performedwhen the video encoder 202 received the recording pause instruction, asa last frame to be contained in the GOP. When the recording pause isreleased, the video encoder pause unit 210 has the video encoder 202start encoding a video frame, which has been first generated after therecording pause release instruction was issued, into video data. As aresult, the video encoder 202 generates an I picture at a start of a newGOP.

[0138] In response to the recording pause instruction, the samplingpause unit 211 has the sampling unit 205 suspend sampling for an audiosignal. At the same instant, the aforementioned last video frame hasbeen generated. When this recording pause is released, the samplingpause unit 211 has the sampling unit 205 resume the sampling whensampling for a video frame is resumed.

[0139] In response to a recording pause instruction, the audio encoderpause unit 212 has the audio encoder 206 suspend generation of audiodata. The audio encoder pause unit 212 also has the audio encoder 206resume generating audio data when the recording pause is released.

[0140] In response to a recording pause instruction, the system encoderpause unit 213 has the system encoder 208 suspend generation of a videopack and an audio pack so that generation of MPEG System stream is alsosuspended. The system encoder pause unit 213 also has the system encoder208 resume generating an audio pack and a video pack to generated MPEGSystem stream when the recording pause is released.

[0141]FIG. 6 shows the processing by the encoding unit 104 when it hasreceived a recording pause instruction during recording, when therecording pause is released, and when the encoding unit 104 has receiveda recoding stop instruction.

[0142] The following describes such processing with reference to FIGS. 5and 6.

[0143] The control I/F unit 201 waits for a user instruction while therecording is performed (step S1).

[0144] On receiving the user instruction, the control I/F unit 201specifies whether the user instruction is a recording pause instructionor a recording stop instruction (step S2).

[0145] If the user instruction is a recording pause instruction, thecontrol I/F unit 201 waits until video signal sampling, which had beenstarted when the recording pause instruction was received, is completedto generate a video frame (step S3).

[0146] As soon as this sampling has been completed, the control I/F unit201 instructs the video encoder pause unit 210, the audio encoder pauseunit 212, and the system encoder pause unit 213 to suspend generation ofvideo data, audio data, and a video pack and an audio pack,respectively. The control I/F unit 201 also instructs the sampling pauseunit 211 to suspend sampling for an audio signal (step S4).

[0147] The video encoder pause unit 210 then has the video encoder 202suspend reception of a video frame. As soon as sampling to generate avideo frame is completed, the sampling pause unit 211 has the samplingunit 205 suspend sampling for the audio signal. The sampling unit 205then holds sound data (hereafter called an audio sub-frame) that is toosmall to form one audio frame during the recording pause (step S5).

[0148] The audio encoder pause unit 212 has the audio encoder 206suspend generation of audio data. The video encoder 202 then receivesthe last video frame, and completes generation of one GOP. The end coderequesting unit 209 then requests the video encoder 202 to generate andinsert a sequence end code into the end of the generated GOP, so thatthe video encoder 202 generates and inserts it (step S6).

[0149] The system encoder pause unit 213 has the system encoder 208suspend generation of a video pack, an audio pack, and MPEG Systemstream (Step S7).

[0150] The control I/F unit 201 waits for a user instruction while therecording is suspended (step S8).

[0151] On receiving the user instruction, the control I/F unit 201specifies whether the user instruction is a recording pause releaseinstruction or a recording stop instruction (step S9).

[0152] When specifying that the user instruction is a recording pauserelease instruction, the control I/F unit 201 waits until a time atwhich the sampling should be resumed to generate a video frame. (stepS10).

[0153] When the time to resume the sampling to generate a video framecomes, the control I/F unit 201 instructs the video encoder pause unit210, the audio encoder pause unit 212, and the system encoder pause unit213 to resume generation of video data, audio data, and video packs andaudio packs, respectively. The control I/F unit 201 also instructs thesampling pause unit 211 to resume sampling an audio signal (step S11)

[0154] In the moment that sampling for a video frame starts, thesampling pause unit 211 has the sampling unit 205 resume sampling anaudio signal so that an audio subframe is generated. The sampling unit205 then joins the generated subframe with the subframe that has beenheld during the recording pause, and outputs the joined subframes as oneaudio frame when the frame generation cycle for audio frames comes.Thereafter, the sampling unit 205 outputs one audio frame correspondingto one sampling cycle in every frame generation cycle. The audio encoderpause unit 212 also has the audio encoder 206 resume generation of audiodata, and the video encoder pause unit 210 has the video encoder 202resume reception of a video frame and generation of video data (stepS12)

[0155] As soon as the video encoder 202 generates the video data, whichis generated first after the recording pause instruction was released,the system encoder pause unit 213 has the system encoder 208 resumegenerating a video pack, an audio pack, and MPEG System stream (stepS13). The control flow them returns to step S1, where a user instructionis awaited.

[0156] On the other hand, if the control I/F unit 201 specifies that theuser instruction is a recording stop instruction in step S2 or S9, thefollowing processing to stop the recording is performed.

[0157] The control I/F unit 201 stops the video encoder 202 fromencoding a video frame as soon as the encoder 202 has completed encodinga video frame which it received immediately before the recording stopinstruction is issued. At the same time, the control I/F unit 201 stopsthe audio encoder 206 from encoding an audio frame as soon as theencoder 206 has completed encoding of an audio frame which it receivedimmediately before the recording stop instruction is issued. The controlI/F unit 201 also has the sampling unit 205 abandon the stored subframe.

[0158] On completing encoding, the video encoder 202 and the audioencoder 206 notifies the control I/F unit 201 of the end of the encodingso that the control I/F unit 201 instructs the system encoder 208 tocomplete generation of MPEG System stream.

[0159] The system encoder 208 completes generation of MPEG Systemstream, using video data and audio data remaining in the video buffer203 and the audio buffer 207.

[0160]FIG. 7 is a timing chart that shows input and output timing ofvideo data and audio data when the recording is suspended and then therecoding suspension is cleared.

[0161] The user instructs to suspend recording at time “ta” in FIG. 7.As a result, sampling for a video signal is suspended at time “tp” assoon as sampling, which was being performed when the user instructed tosuspend the recording, has been completed and a video frame has beengenerated. Simultaneously, sampling for an audio signal is alsosuspended.

[0162] The user then instructs to release the recording pause at time“tb”. As a result, sampling for a video signal is resumed at time “tr”that is in a sampling cycle to generate a first video frame after therecording pause is released. Simultaneously, sampling for an audiosignal is resumed. Accordingly, the input of AV data is suspended for aperiod between time “tp” and time “tr”, and this period is hereaftercalled an “AV data input pause period”.

[0163] The video encoder 202 suspends generation of video data as soonas it has completes encoding of a video frame which it has received atthe time of the reception of the recording pause instruction. When aperiod that has the same duration as the aforementioned AV data inputpause period has passed, the suspension of generation of video data iscleared as the video encoder 202 resumed encoding a video frame which itfirstly received after the recording pause was cleared. The videoencoder 202 holds the VBV buffer transition information during a periodfor which it suspends generation of video data, and uses the held VBVbuffer transition information when resuming generation of video data.

[0164] The audio encoder 206 suspends generation of audio data in themoment it completes encoding of an audio frame (not an audio subframe)which it has received at the time of the reception of the recordingpause instruction. When a period that has the same duration as the AVdata input pause period has passed, the audio encoder 206 resumesencoding an audio frame which it has firstly received after therecording pause is released.

[0165] A uniform time is taken to perform sampling for an audio signalto generate an audio frame and encode the audio frame into audio data.As a result, a period for which the sampling for an audio signal issuspended becomes equal to a period for which the audio encoder 206suspends generation of audio data.

[0166] The following describes how video data and audio data are heldwhile recording is suspended.

[0167]FIG. 8 shows, as one example, how video data and audio data areused when the recording pause instruction is issued.

[0168] As shown in the figure, at time “tp1”, generation of MPEG Systemstream is suspended.

[0169] The system encoder 208 continues to generate MPEG System streameven after the video encoder 202 has suspended generation of video data.The system encoder 208 suspends generation of MPEG System stream as soonas it has multiplexed all the video data remaining in the video buffer207. On the other hand, audio data buffered in the audio buffer 207 isnot multiplexed into the MPEG System stream, and held in the audiobuffer 207 while the recording is suspended.

[0170]FIG. 9 shows how video data and audio data are used as oneexample.

[0171] As shown in the figure, generation of MPEG System stream issuspended during a period that has the same duration as a period forwhich AV data input is suspended. As a result, a time lag between animage and sound, which are recorded after the recording pause can beprevented from being generated.

[0172] Note that a timing with which the system encoder 208 suspendsgeneration of MPEG System stream is not limited to the above examples.For instance, the system encoder 208 may suspend generation of MPEGSystem stream as soon as the video encoder 202 has suspended generationof video data, or before the system encoder 208 completes multiplexingof all the video data remaining in the video buffer 203.

[0173] Also, a timing with which the audio encoder 206 suspendsgeneration of audio data is not limited to the above examples. Forinstance, the audio encoder 206 may suspend generation of audio data assoon as the video encoder 202 has suspended generation of video data, orbefore encoding all the audio frames which have been obtained beforesampling was completed for a video signal, for which sampling was beingperformed when the recording pause instruction was issued.

[0174]FIG. 10 shows, as another example, how video data and audio datagenerated by the video encoder 202 and the audio encoder 206 are used.

[0175] As shown in the figure, the system encoder 208 suspendsgeneration of MPEG System stream at time “tp2” after time “Δt” haspassed since the video encoder 202 suspended generation of video data.This time “Δt” is defined in a local standard for the present DVDrecorder, and has the same duration as one video frame generation cyclein this example.

[0176] The system encoder 208 continues to generate MPEG System streameven after the video encoder 202 has suspended generation of video data,and suspends generation of the stream at time tp2, which is before itfinishes multiplexing all the video data remaining in the video buffer203. The generated video data and audio data that have not beenmultiplexed remain to be stored in the video buffer 203 and the audiobuffer 207 while the recording is suspended.

[0177] The audio encoder 206 suspends generation of audio data beforefinishing encoding all the audio frames which it has received before therecording pause instruction was issued.

[0178] The following describes how video data and audio data are usedafter the recording pause is released.

[0179]FIG. 11 shows, as one example, how video data and audio data areused before and after the recording pause.

[0180] As shown in the figure, sampling for the video signal and theaudio signal is suspended for the same period. In addition, all thevideo data and audio data generated by encoding the video signal and theaudio signal which were sampled before and after the recording pause ismultiplexed into an MPEG System stream. This can prevent a time lag frombeing generated between images and sound which are recorded before andafter the recording pause.

[0181] Also, the system encoder 208 suspends generation of MPEG Systemstream when about one video frame generation cycle has passed since thevideo encoder 202 suspended generation of video data. The system encoder208 then resumes generating MPEG System stream when about one videoframe generation cycle has passed since the video encoder 202 resumedgeneration of video data. As a result, the system encoder 208 suspendsgeneration of MPEG System stream for a period having the same durationas a period for which input of AV data is suspended.

[0182]FIG. 12 shows how video data and audio data are used when arecording stop instruction is issued during recording.

[0183] As shown in the figure, in response to the recording stopinstruction issued at time “tc”, sampling for a video signal stops at“ts” as soon as sampling, which was being performed at time “tc” togenerate a video frame, has been completed.

[0184] The video encoder 202 stops generating video data when it hasencoded all the video frames which it received before sampling for avideo signal was suspended.

[0185] The system encoder 208 continues to generate MPEG System streameven after the video encoder 202 stops generating video data. Onmultiplexing all the video data remaining in the video buffer 203, thesystem encoder 208 stops the multiplexing. The system encoder 208 alsomultiplexes audio data, which has been generated by the audio encoder206 using audio frames generated before the time “ts”, into the MPEGSystem stream.

[0186] Accordingly, when recording operation stops in response to therecording stop instruction, the resulting MPEG System stream contains asequence of audio packs in the end of the stream.

[0187] As soon as the system encoder 208 stops the multiplexing, thesampling unit 205 and the audio encoder 206 stop operating. The audioencoder 206 discards audio data generated from audio frames, which havebeen generated after time “ts”.

[0188]FIG. 13 shows how video data and audio data are used when arecording stop instruction is issued while recording is suspended asshown in FIG. 8.

[0189] When the user instructs to stop the recording at time “td”, thesystem encoder 208 receives the recording stop instruction at time“ts1”, and starts multiplexing all the audio data remaining in the audiobuffer 207 into MPEG System stream, and completes the multiplexing. Theresulting MPEG System stream contains a sequence of audio packs in theend of the stream.

[0190]FIG. 14 shows how video data and audio data are used when arecording stop instruction is issued while recording is suspended asshown in FIG. 10.

[0191] When the user instructs to stop the recording at time “te”, thesystem encoder 208 receives the recording stop instruction at time“ts2”, and then resumes multiplexing using all the audio data and videodata remaining in the audio buffer 207 and the video buffer 203.

[0192] The audio encoder 206 also resumes generating audio data. Thesystem encoder 208 multiplexes all the audio data, which has beengenerated by the audio encoder 206 using audio frames which it receivedbefore suspending generation of audio data, into the end of the MPEGSystem stream, and completes the multiplexing.

[0193] Video data recorded after the recording pause often correspondsto the start of a program or a take, and therefore the user may searchfor this certain video data more often than he searches for other videodata for reproduction. For DVD-RAM, address search within a VOB isperformed using Video Object Unit (VOBU) Time Map Information (TMAPI).Accordingly, it is desirable to insert a video pack, which contains thestart of a GOP generated immediately after the recording pause has beenreleased, into a VOBU as the first video pack in order to allow the userto effectively search for video data recorded after the recording pauseis released.

[0194] Accordingly, when the user instructs to suspend the recording,the system encoder 208 completes generation of one VOBU using videodata, which the video encoder 202 has generated using a video signalinputted before the recording suspend instruction is issued, and usingaudio data which is multiplexed before this video data. When the userinstructs to release the suspension and the system encoder 202 resumesgeneration of packs, the system encoder 202 inserts a video pack, whichcontains the start of video data generated by the video encoder 202immediately after the suspension is released, into the start of a VOBU.

[0195] Here, a VOB is MPEG System stream and is composed of a pluralityof VOBUs, and each VOBU is a set of data that has a construction of MPEGSystem stream into which at least one GOP and a plurality of audio packsare multiplexed.

[0196] However, recording AV data before and after the recording pauseas different VOBUs involves the following problem in relation to theDVD-RAM standard.

[0197] Video Object Information (VOBI) defines an entry “VOB#REC#Time”,and a recording start time for one VOB is recorded in this entry whichis present at a start of the VOBI. Accordingly, when different pieces ofstream before and after a recording pause are recorded as one VOB, thesepieces of stream have the same “VOB#REC#Time”. However, this is notdesirable as these pieces of stream are not recorded in one continuousperiod.

[0198] Accordingly, the system encoder 208 completes generation of oneVOB when suspending generation of packs in response to a recordingsuspend instruction. When resuming generation of packs in response to arecording pause release instruction, the system encoder 208 inserts aVOBU generated immediately after the suspension is released into a newVOB, and records a time at which the suspension was released into thestart of the new VOB as the entry “VOB#REC#Time”.

[0199] It is also defined by the DVD-RAM standard that a sequence endcode “0x000001B7 (hexadecimal)” should be inserted into the end of a GOPgenerated immediately before a recording pause if AV data before andafter the recording pause is recorded as different VOBs. This sequenceend code shows the end of MPEG System video stream. An ordinary videoencoder has a function to generate a sequence end code.

[0200] The end code requesting unit 209 therefore instructs the videoencoder 202 to generate a sequence end code so that the video encoder202 generates a sequence end code and inserts it into the end of a GOPgenerated immediately before the recording pause.

[0201]FIG. 15 shows a sequence end code inserted into the end of a GOPgenerated immediately before the suspension.

[0202] As shown in the figure, the video encoder 202 regards thesequence end code as a part of the GOP (video data) when counting agenerated bit size. As a result, the system encoder 208 does not need toconsider a bit size (32 bits) of the sequence end code.

[0203] Note that the above generation and insertion of a sequence endcode do not have to be performed by the video encoder 202, and may beperformed by the system encoder 208, for instance.

[0204]FIG. 16 shows a state in which the system encoder 208 inserts asequence end code into the end of a GOP generated immediately before arecording pause.

[0205] As can be understood from the figure, the end code requestingunit 209 is no longer necessary in this case although it is necessaryfor the system encoder 208 to consider an increase by the bit size (32bits) of the sequence end code when generating packs. In some cases, anew pack for storing the sequence end code may have to be generated.

[0206] As has been described, on receiving a recording pauseinstruction, the encoding/recording device of the present inventionsuspends operations of the video encoder with a timing conforming to avideo frame generation cycle. When the recording pause is released, theencoding/recording device has the video encoder resume the operations.This video encoder generates video data while considering transition ina size of data accumulated in a video buffer of an MPEG decoder in everyvideo frame generation cycle. Accordingly, video data recorded by thepresent encoding/recording device can be continuously reproduced withoutthe video buffer in the MPEG decoder being damaged.

[0207] In addition, the present encoding/recording device suspendsoperations of the video encoder simultaneously with sampling for anaudio signal. When having the video encoder resume its operations, theencoding/recording device simultaneously has sampling for an audiosignal resumed. As a result, audio data recorded by thisencoding/recording device can be reproduced without a time lag betweenimages and sound being generated.

[0208] The above embodiment has been used to describe the presentinvention although it should be clear that the present invention is notlimited to these embodiments. This is to say, computer programs to havea computer perform the processing of the above embodiment may berecorded on computer-readable recording media to be distributed andsold. These recording media can be, for instance, a floppy disc, acompact disc, a magnet-optical disc, a DVD disc, and a memory card, allof which allow the user to load into and remove from a computer. Therecording media may be also hard disk, semiconductor memory, or the likethat are provided in a computer in advance.

[0209] Although the present invention has been fully described by way ofexamples with reference to accompanying drawings, it is to be noted thatvarious changes and modifications will be apparent to those skilled inthe art. Therefore, unless such changes and modifications depart fromthe scope of the present invention, they should be construed as beingincluded therein.

What is claimed is:
 1. An encoding/recording device that receives andencodes a video signal and an audio signal, multiplexes the encodedvideo signal and audio signal to produce a system stream, and recordsthe system stream on a recording medium, the encoding/recording devicecomprising: a video data generating unit operable to (a) estimate anamount of data, which occupies a buffer in a decoder when the decoderdecodes the encoded video signal, (b) store how the amount of datachanges over time as buffer information, (c) receive a video signal, (d)encode the received video signal to generate video data in a manner thatprevents the decoder from overflowing and underflowing, and (e) updatethe buffer information whenever video data is generated; an audio datagenerating unit operable to receive an audio signal, and encode eachpart of the received audio signal to generate audio data, each parthaving a predetermined size; a multiplexing/recording unit operable tomultiplex the generated video data and audio data, and record themultiplexed video data and audio data onto the recording medium; a pausecontrolling unit operable to specify a pause timing on receiving a pauseinstruction that suspends recording by the encoding/recording devicefrom a user during the recording, and have the video data generatingunit, the audio data generating unit, and the multiplexing/recordingunit suspend operations; and a pause release controlling unit operableto specify a pause release timing on receiving a pause releaseinstruction from the user, and have the video data generating unit, theaudio data generating unit, and the multiplexing/recording unit resumethe operations, wherein under control of the pause controlling unit, thevideo data generating unit suspends reception of the video signal withthe specified pause timing without resetting the stored bufferinformation and under control of the pause release controlling unit, thevideo data generating unit resumes the reception the video signal withthe specified pause release timing and resumes encoding the receivedvideo signal based on the stored buffer information, wherein undercontrol of the pause controlling unit, the audio data generating unitsuspends reception of the audio signal with the specified pause timingand under control of the pause release controlling unit, the audio datagenerating unit resumes receiving the audio signal with the specifiedpause release timing, and resumes encoding the received audio signal,and wherein the multiplexing/recording unit suspends multiplexing andrecording with a predetermined timing that is equal to or later than thespecified pause timing under control of the pause controlling unit, andresumes the multiplexing and the recording with a predetermined timingthat is equal to or later than the specified pause release timing undercontrol of the pause release controlling unit.
 2. The encoding/recordingdevice of claim 1, wherein the video data generating unit encodes thereceived video signal according to MPEG (Moving Picture Expert Group).3. The encoding/recording device of claim 1, further comprising a timerthat generates a reference time at least when the encoding/recordingdevice operates and suspends recording, wherein every first predetermineperiod, the video data generating unit receives a video signal thatcorresponds to one video frame, each first predetermined period beingbased on a reference time generated by the timer, wherein the pausetiming specified by the pause controlling unit is synchronous to aboundary of two first predetermined periods, and wherein the pauserelease timing specified by the pause release controlling unit issynchronous to a boundary of two first predetermined periods.
 4. Theencoding/recording device of claim 3, wherein under control of the pausecontrolling unit, the multiplexing/recording unit suspends themultiplexing and the recording after a second predetermined period haspassed since the specified pause timing, and holds video data and audiodata which have not been multiplexed and recorded, and under control ofthe pause release controlling unit, the multiplexing/recording unitresumes the multiplexing and the recording by multiplexing and recordingthe held video data and audio data after the second predetermined periodhas passed since the specified pause release timing.
 5. Theencoding/recording device of claim 4, wherein after a thirdpredetermined period has passed since the specified pause timing, thevideo data generating unit completes encoding for the video signal whichhas been received before the specified pause timing and wherein thesecond predetermined period for the multiplexing/recording unit is a sumof the first predetermined period and the third predetermined period. 6.The encoding/recording device of claim 5, further comprising a firststop controlling unit operable to specify a stop timing on receiving astop instruction from the user while the encoding/recording deviceoperates, and to have the video data generating unit, the audio datagenerating unit, and the multiplexing/recording unit stop operations,wherein under control of the first stop controlling unit, the video datagenerating unit stops receiving the video signal with the specified stoptiming, and encodes the video signal received before the stop timing togenerate video data, wherein under control of the first stop controllingunit, the audio data generating unit stops receiving the audio signalwith the specified stop timing, encodes each part of the predeterminedsize contained in the received audio signal to generate audio data, andabandons, if the received audio signal contains a subpart that issmaller than the predetermined size, the subpart and wherein undercontrol of the first stop controlling unit, the multiplexing/recordingunit multiplexes and records all the audio data and video data that havebeen generated.
 7. The encoding/recording device of claim 6, furthercomprising: a second stop controlling unit operable to specify a stoptiming on receiving a stop instruction from the user while theencoding/recording device suspends recording in response to a recordingpause instruction, and to have the video data generating unit, the audiodata generating unit, and the multiplexing/recording unit stopoperations, wherein under control of the second stop controlling unit,the video data generating unit encodes the video signal received beforethe specified pause timing to generate video data, wherein under controlof the second stop controlling unit, the audio data generating unitencodes each part of the audio signal received before the specifiedpause timing to generate audio data, and abandons, if the received audiosignal contains a subpart, the subpart and wherein under control of thesecond stop controlling unit, the multiplexing/recording unitmultiplexes and records the held video data and audio data, and thegenerated audio data and video data.
 8. The encoding/recording device ofclaim 7, wherein the audio data generating unit contains: an audiosignal sampling unit operable to receive the audio signal, sample thereceived audio signal to generate sets of sampled data that eachcorrespond to one audio sampling cycle, each set of sampled data beinggenerated every fourth predetermined period that is different from thefirst predetermined period, and suspend reception of the audio signalwith the specified pause timing; an encoding unit operable to receive aset of sampled data from the audio signal sampling unit every fourthpredetermined period, and encode the received set of sampled data togenerate audio data corresponding to one audio sampling cycle; afractional data holding unit operable to hold a set of sampled datawhich corresponds to a time shorter than the fourth predetermined periodwhen the audio signal sampling unit suspends the reception of the audiosignal; and wherein the audio signal sampling unit resumes receiving theaudio signal with the specified pause release timing, samples thereceived audio signal to generate a set of sampled data, joins thegenerated set of sampled data and the held set of sampled data togetherto produce a set of sampled data corresponding to one audio samplingcycle, outputs the produced set of sampled data, and thereafter outputsa set of sampled data corresponding to one audio sampling cycle everyfourth predetermined period.
 9. The encoding/recording device of claim3, wherein under control of the pause controlling unit, themultiplexing/recording unit suspends the multiplexing and the recordingafter having multiplexed and recorded all the video data generated bythe video data generating unit.
 10. The encoding/recording device ofclaim 9, wherein when suspending the multiplexing and the recording, themultiplexing/recording unit holds audio data that has not beenmultiplexed and recorded, and wherein under control of the pause releasecontrolling unit, the multiplexing/recording unit resumes themultiplexing and the recording by starting multiplexing and recording(a) the held audio data and (b) video data generated after the specifiedpause release timing.
 11. The encoding/recording device of claim 10,further comprising a first stop controlling unit operable to specify astop timing on receiving a stop instruction from the user while theencoding/recording device operates, and to have the video datagenerating unit, the audio data generating unit, and themultiplexing/recording unit stop operations, wherein under control ofthe first stop controlling unit, the video data generating unit stopsreceiving the video signal with the specified stop timing, and encodesthe video signal received before the stop timing to generate video data,wherein under control of the first stop controlling unit, the audio datagenerating unit stops receiving the audio signal with the specified stoptiming, encodes each part of the predetermined size contained in thereceived audio signal to generate audio data, and abandons, if thereceived audio signal contains a subpart that is smaller than thepredetermined size, the subpart and wherein under control of the firststop controlling unit, the multiplexing/recording unit multiplexes andrecords all the audio data and video data that have been generated. 12.The encoding/recording device of claim 11, wherein themultiplexing/recording unit further comprising a second stop controllingunit operable to specify a stop timing on receiving a stop instructionfrom the user while the encoding/recording device suspends recording,and to have the video data generating unit, the audio data generatingunit, and the multiplexing/recording unit stop operations, wherein theaudio data generating unit abandons a subpart if the received audiosignal contains the subpart, and wherein under control of the secondstop controlling unit, the multiplexing/recording unit multiplexes andrecords all the held audio data.
 13. The recording/encoding device ofclaim 12, wherein the audio data generating unit contains: an audiosignal sampling unit operable to receive the audio signal, sample thereceived audio signal to generate sets of sampled data that eachcorrespond to one audio sampling cycle, each set of sampled data beinggenerated every fourth predetermined period that is different from thefirst predetermined period, and suspend reception of the audio signalwith the specified pause timing; an encoding unit operable to receive aset of sampled data from the audio signal sampling unit every fourthpredetermined period, and encode the received set of sampled data togenerate audio data corresponding to one audio sampling cycle; afractional data holding unit operable to hold a set of sampled datawhich corresponds to a time shorter than the fourth predetermined periodwhen the audio signal sampling unit suspends the reception of the audiosignal; and wherein the audio signal sampling unit resumes receiving theaudio signal with the specified pause release timing, samples thereceived audio signal to generate a set of sampled data, joins thegenerated set of sampled data and the held set of sampled data togetherto produce a set of sampled data corresponding to one audio samplingcycle, outputs the produced set of sampled data, and thereafter outputsa set of sampled data corresponding to one audio sampling cycle everyfourth predetermined period.
 14. The encoding/recording device of claim1, wherein the audio data generating unit contains a muting unitoperable to lower a sound intensity of the audio signal before receptionof the audio signal is suspended, and to restore the sound intensitybefore the reception of the audio signal is resumed.
 15. Theencoding/recording device of claim 2, wherein the video signal receivedby the video data generating unit corresponds to a plurality of videoframes, wherein the video data generating unit encodes the receivedvideo signal using time correlation properties between some of theplurality of video frames, and generates a plurality of GOPs (groups ofpictures), wherein each GOP is video data corresponding to a pluralityof video frames that are encoded with reference only to video frames inthe same GOP, and wherein when suspending the reception of the videosignal, the video data generating unit completes generation of a GOP.16. The encoding/recording device of claim 15, wherein themultiplexing/recording unit generates system stream by multiplexingaudio data and video data, the system stream being composed of aplurality of video object units (VOBUs), wherein each VOBU is composedof at least one GOP and audio data related to the at least one GOP, andhas a representation time shorter than a predetermined time, whereinwhen suspending the recording, the multiplexing/recording unit completesgeneration of a VOBU using video data, which has been generated by thevideo data generating unit using the video signal received before thespecified pause timing, and audio data corresponding to a decoding orderthat is earlier than the decoding order corresponding to the used videodata, wherein the decoding order is an order in which the decoderdecodes the video data and the audio data, wherein when resuming therecording, the multiplexing/recording unit makes video data, which hasbeen generated by the video data generating unit immediately after thespecified pause release timing, video data placed first in a VOBU thatfollows the VOBU generated when the recording is suspended.
 17. Theencoding/recording device of claim 16, wherein themultiplexing/recording unit generates a video object (VOB) composed of aplurality of VOBUs, the VOB containing a recording region into which arecording start time for the VOB should be written, wherein whensuspending the recording, the multiplexing/recording unit completesgeneration of a VOB, and wherein when resuming the recording, themultiplexing/recording unit newly generates a VOBU and a VOB, places thegenerated VOBU at a start of the generated VOB, and writes a time atwhich the recording is resumed as a recording start time into arecording region in the generated VOB.
 18. The encoding/recording deviceof claim 15, wherein when suspending the reception of the video signal,the video data generating unit attaches a sequence end code to an end ofvideo data which has been generated from the received video signal, thesequence end code indicating an end of video stream that is video datacorresponding to a plurality of video frames.
 19. The encoding/recordingdevice of claim 15, wherein when suspending the recording, themultiplexing/recording unit attaches a sequence end code to an end ofvideo data, which has been generated by the video data generating unitusing the video signal received before the specified pause timing, thesequence end code indicating an end of video stream that is video datacorresponding to a plurality of video frames.
 20. An encoding/recordingmethod for receiving and encoding a video signal and an audio signal,multiplexing the encoded video signal and audio signal to produce asystem stream, and recording the system stream on a recording medium,the encoding/recording method including: a video data generating stepfor (a) estimating an amount of data, which occupies a buffer in adecoder when the decoder decodes the encoded video signal (b) storinghow the amount of data changes over time as buffer information, (c)receiving a video signal, (d) encoding the received video signal togenerate video data in a manner that prevents the decoder fromoverflowing and underflowing, and (e) updating the buffer informationwhenever video data is generated; an audio data generating step forreceiving an audio signal and encoding each part of the received audiosignal to generate audio data, each part having a predetermined size; amultiplexing/recording step for multiplexing the generated video dataand audio data, and recording the multiplexed video data and audio dataonto the recording medium; a pause controlling step for specifying apause timing on receiving a pause instruction from a user during therecording, and having operations in the video data generating step, theaudio data generating step, and the multiplexing/recording stepsuspended; and a pause release controlling step for specifying a pauserelease timing on receiving a pause release instruction from the user,and having the operations in the video data generating step, the audiodata generating step, and the multiplexing/recording step resumed,wherein in the video data generating step under control of operations inthe pause controlling step, reception of the video signal is suspendedwith the specified pause timing without the stored buffer informationbeing reset and under control of operations in the pause releasecontrolling step, the reception the video signal is resumed with thespecified pause release timing, and encoding for the received videosignal is resumed based on the stored buffer information, wherein in theaudio data generating step under control of-operations in the pausecontrolling step, reception of the audio signal is suspended with thespecified pause timing and under control of operations in the pauserelease controlling step, the reception of the audio signal is resumedwith the specified pause release timing, and encoding of the receivedaudio signal is resumed and wherein in the multiplexing/recording stepunder control of operations in the pause controlling step, multiplexingand recording are suspended with a predetermined timing that is equal toor later than the specified pause timing and under control of operationsin the pause release controlling step, the multiplexing and therecording are resumed with a predetermined timing that is equal to orlater than the specified pause release timing.
 21. Theencoding/recording method of claim 20, further including a clocking stepfor keeping generating a reference time regardless of whether operationsare performed or suspended, wherein in the video data generating step, avideo signal that corresponds to one video frame is received every firstpredetermine period, each first predetermined period being based on areference time generated in the clocking step, wherein the pause timingspecified in the pause controlling step is synchronous to a boundary oftwo first predetermined periods, and wherein the pause release timingspecified in the pause release controlling step is synchronous to aboundary of two first predetermined periods.
 22. The encoding/recordingmethod of claim 20, wherein the audio data generating step contains amuting step for lowering a sound intensity of the audio signal beforereception of the audio signal is suspended, and restoring the soundintensity before the reception of the audio signal is resumed.
 23. Arecording medium storing a program to have a computer perform anencoding/recording method for receiving and encoding a video signal andan audio signal, multiplexing the encoded Video signal and audio signalto produce a system stream, and recording the system stream on arecording medium, the encoding/recording method including: a video datagenerating step for (a) estimating an amount of data, which occupies abuffer in a decoder when the decoder decodes the encoded video signal(b) storing how the amount of data changes over time as bufferinformation, (c) receiving a video signal, (d) encoding the receivedvideo signal to generate video data in a manner that prevents thedecoder from overflowing and underflowing, and (e) updating the bufferinformation whenever video data is generated; an audio data generatingstep for receiving an audio signal and encoding each part of thereceived audio signal to generate audio data, each part having apredetermined size; a multiplexing/recording step for multiplexing thegenerated video data and audio data, and recording the multiplexed videodata and audio data onto the recording medium; a pause controlling stepfor specifying a pause timing on receiving a pause instruction from auser during the recording, and having operations in the video datagenerating step, the audio data generating step, and themultiplexing/recording step suspended; and a pause release controllingstep for specifying a pause release timing on receiving a pause releaseinstruction from the user, and having the operations in the video datagenerating step, the audio data generating step, and themultiplexing/recording step resumed, wherein in the video datagenerating step under control of operations in the pause controllingstep, reception of the video signal is suspended with the specifiedpause timing without the stored buffer information being reset and undercontrol of operations in the pause release controlling step, thereception the video signal is resumed with the specified pause releasetiming, and encoding for the received video signal is resumed based onthe stored buffer information, wherein in the audio data generating stepunder control of operations in the pause controlling step, reception ofthe audio signal is suspended with the specified pause timing and undercontrol of operations in the pause release controlling step, thereception of the audio signal is resumed with the specified pauserelease timing, and encoding of the received audio signal is resumed andwherein in the multiplexing/recording step under control of operationsin the pause controlling step, multiplexing and recording are suspendedwith a predetermined timing that is equal to or later than the specifiedpause timing and under control of operations in the pause releasecontrolling step, the multiplexing and the recording are resumed with apredetermined timing that is equal to or later than the specifiedpause-release timing.
 24. The recording medium of claim 23, wherein theencoding/recording method further including a clocking step for keepinggenerating a reference time regardless of whether operations areperformed or suspended, wherein in the video data generating step, avideo signal that corresponds to one video frame is received every firstpredetermine period, each first predetermined period being based on areference time generated in the clocking step, wherein the pause timingspecified in the pause controlling step is synchronous to a boundary oftwo first predetermined periods, and wherein the pause release timingspecified in the pause release controlling step is synchronous to aboundary of two first predetermined periods.
 25. The recording medium ofclaim 23, wherein the audio data generating step contains a muting stepfor lowering a sound intensity of the audio signal before reception ofthe audio signal is suspended, and restoring the sound intensity beforethe reception of the audio signal is resumed.